Semi-supervised Learning

Semi-supervised learning uses both labeled and unlabeled data to perform an otherwise supervised learning or unsupervised learning task. In the former case, there is a distinction between inductive semi-supervised learning and transductive learning. In inductive semi-supervised learning, the learner has both labeled training data {(xi, yi)}i=1 iid ∼ p(x, y) and unlabeled training data {xi} i=l+1 iid ∼ p(x), and learns a predictor f : X 7→ Y, f ∈ F where F is the hypothesis space. Here x ∈ X is an input instance, y ∈ Y its target label (discrete for classification or continuous for regression), p(x, y) the unknown joint distribution and p(x) its marginal, and typically l u. The goal is to learn a predictor that predicts future test data better than the predictor learned from the labeled training data alone. In transductive learning, the setting is the same except that one is solely interested in the predictions on the unlabeled training data {xi} i=l+1, without any intention to generalize to future test data. In the latter case, an unsupervised learning task is enhanced by labeled data. For example, in semi-supervised clustering (a.k.a. constrained clustering) one may have a few must-links (two instances must be in the same cluster) and cannot-links (two instances cannot be in the same cluster) in addition to the unlabeled instances to be clustered; in semi-supervised dimensionality reduction one might have the target low-dimensional coordinates on a few instances. This entry will focus on the former case of learning a predictor.